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motivation 


■ need for Human — Automation Interaction (HAI) test support in 
the aircraft certification and approval process 

■ existing formal method algorithms and framework might help 

■ but any results must be transparent and usable by evaluator 


automated test-case generation through 

symbolic execution 
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concept 



for ml = 1,M do begin 
for m2 = 1,M do begin 
for ul = u_min,u_max do begin 
for u2 = u_min, u_max do begin 
if ul gt u2 then begin 
for vl = v_min,v_max do begin 
if vl It ul then begin 
for v2 = v_min,v_max do begin 
if v2 ge vl then begin 
KE_B = double(ml*ul A 2+m2*u2 / '2 ) 

KE_A = double (ml*vl , '2+m2*v2* 2 ) 

if (KE_B gt KE_A) and (KE_A ge 0.965*KE_B) then 1 
x_axis[index]=index 
IM_B = double(ml*ul+m2*u2) 

IM_A = double(ml*vl+m2*v2) 
y_IM_Di f f s [ index ] =LM_B-LM_A 
Total_LM=Total_IM+IM_B-IM_A 

y_LM_Total [ index] =double(Total_LM/( index+1 ) ) 
index=index+l 

if index gt 65535 then goto, end_of_loop 
end if 
end if 
end for 
end if 
end for 
endif 
end for 
endf or 
end for 
endf or 


source code 
(main method) 



symbolic execution to 
derive execution paths 
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Usability Test 







why symbolic execution? 


(^Symbolic ("true") 
int x; 

(^Symbolic ("true") 
int y; 


void testXQ { 

if (x > 0) 

y = y + x; 

else 


} 
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why symbolic execution? 


(^Symbolic ("true") 
int x; 

(^Symbolic ("true") 
int y; 


void testXQ { 

if (x > 0) 

y = y + x; 

else 


} 




x:X, y : Y 
PC: X > 0 

I 

x:X, y :Y+X 
PC: X > 0 


x:X, y : Y 
PC: X <= 0 

I 

x:X, y :Y-X 
PC: X <= 0 
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why symbolic execution? 


(^Symbolic ("true") 
int x; 

(^Symbolic ("true") 
int y; 


void tesjtXQ { 

if (x > 0) 

y = y + x; 

else 


} 



x:X, y : Y 
PC: true 


x:X, 
PC: ) 

\ 

y : Y 
( > 0 

x:X, y : Y 
PC: X <= 0 

> 

f 

IJZ 

x:X, 

y :Y+X 

x:X, y :Y-X 

PC: X > 0 

PC: X <= 0 
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why symbolic execution? 


(^Symbolic ("true") 
int x; 

(^Symbolic ("true") 
int y; 

void testXQ { 

if (x > 0) 

+ x; 


- x; 

} 


y = y 

else 

y = y 


x:X, y : Y 
PC: true 
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why symbolic execution? 


(^Symbolic ("true") 
int x; 

@Symbolic("true") 
int y; 

void testXQ { 

if (x > 0) 

C 


y = y + x; 


else 


y = y - x; 


} 


x:X, y : Y 
PC: true 



x:X, 
PC: ) 

y : Y 
( > 0 

x:X, y : Y 
PC: X <= 0 

> 

f 



\ 


x:X, 

y :Y+X 

x:X, y :Y-X 

PC: X > 0 

J 

PC: X <= 0 
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why symbolic execution? 


(^Symbolic ("true") 
int x; 

(^Symbolic ("true") 
int y; 

void testXQ { 

if (x > 0) 

y = y + x; 

else 

y = y -~xj 

} 


x:X, y : Y 
PC: true 


x:X, y : Y 

x:X^ 

y : Y 

PC: X > 0 

PC: X 

<= 0 

irz 

> 

f 



\ 

x:X, y :Y+X 

x:X, 

y : Y-X 

PC: X > 0 

PC: X <= 0 



J 



Federal Aviation 
Administration 




why symbolic execution? 


(^Symbolic ("true") 
int x; 

(^Symbolic ("true") 
int y; 


void testXQ { 

if (x > 0) 

y = y + x; 

else 


} 



x:X, y : Y 
PC: true 



x:X, y : Y 
PC: X > 0 

I 

x:X, y :Y+X 
PC: X > 0 


x:X, y : Y 
PC: X <= 0 

I 

x:X, y :Y-X 
PC: X <= 0 


Test Input Generation 


X = 1 


X = 0 



Federal Aviation 
Administration 




...when successful, automated test case generation 
automatically generates high quality test suites for full 
path coverage 



Step I: ADEPT to Java 


Autopilot Example 


in- 


TEST j J © J >H >B V A fi] [fi 
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lateralTarget 
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Capture and ...plan Target 
Sffil selected LateralTargetError 
>179 
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Behavior 











isNominal 











True 


• 

• 

• 

• 

• 

h 

h 


• 

False 







• 

• 

• 


Inputs 











simulationStatus 











paused 

• 










running 


• 

• 

• 







lateral Interface Action OutputState 









J 


noAction 

• 

• 

• 

• 






— 

user presses Lateral Target knob 





• 

• 

• 

• 



user presses Lateral Hold button 









• 


user presses LNAV button 










• 

lateral system table output state 











caputure and maintain selected lateral target 


• 



• 






hold selected lateral target 



• 



• 

• 

• 



capture and maintain lateral flight plan 




• 


• 

• 

• 



selected lateral target error 











> 179 







• 




<= 179&& >= -179 






• 





< -179 








• 



Outputs 









lateral system table output state 











caputure and maintain selected lateral target 





• 

• 

• 

• 



hold selected lateral target 









• 


capture and maintain lateral flight plan 










• 

selected lateral target error 











- = 360 
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+ = 360 
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0 
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preselected lateral target 











lateral direction 



Z 
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selected lateral Target 
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preselected lateral target 





• 

• 

• 

• 



lateral direction 
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lateral target 











selected lateral target 


• 



• 

• 

• 

• 



lateral direction 



• 
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lateral flight plan target 




• 






• 

lateral target error 











selected lateral target error 






• 

• 

• 



lateral flight plan target error 










• 

0 
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□ 


_ 

• 



if ( ! isNominal && ( (outputState == 1) || 
(outputstate == 2)) && 
selectedLateralTargetError > 179 && 
(userPressesLateralTargetButton == true && 
userPressesLateralHoldButton == false && 
userPressesLNAVbutton == false) ){ 
applyRule06(); 

} 

if (! isNominal &&( (outputstate == 1) || 
(outputstate == 2)) && 
selectedLateralTargetError < -179 && 
(userPressesLateralTargetButton == true && 
userPressesLateralHoldButton == false && 
userPressesLNAVbutton == false) ){ 
applyRule07(); 


public void applyRule06( ) { 
outputstate = 0; 

selectedLateralTargetError -= 360; 
selectedLateralTarget = 

preSelectedLateralTarget; 
lateralTarget = selectedLateralTarget; 
lateralTargetError = 

selectedLateralTargetError; 

} 

public void applyRule07( ) { 
outputstate = 0; 

selectedLateralTargetError += 360; 
selectedLateralTarget = 

preSelectedLateralTarget; 
lateralTarget = selectedLateralTarget; 
lateralTargetError = 

selectedLateralTargetError; } 





Step 2: Symbolic Execution 


what do we execute symbolically? 


■ method execute - parameters are user inputs (eg button 
presses) and are symbolic 

■ other (not user input) variables in the table that appear in rule 
conditions are eligible to be treated as symbolic; this allows us to 
explore different initial values that may lead us to different paths 

■ the main method calls method execute n times (n can be 
selected); each time, fresh values are picked for the symbolic 
parameters since each time the user input actions may vary 
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if ( ! isNominal && ( (outputState == 1) || 
(outputState == 2)) && 
selectedLateralTargetError > 179 && 
(userPressesLateralTargetButton == true && 
userPressesLateralHoldButton == false && 
userPressesLNAVbutton == false) ){ 
applyRule06(); 

} 

if (! isNominal &&( (outputState == 1) || 
(outputState == 2)) && 
selectedLateralTargetError < -179 && 
(userPressesLateralTargetButton == true && 
userPressesLateralHoldButton == false && 
userPressesLNAVbutton == false) ){ 
applyRule07(); 


public void applyRule06() { 
outputState = 0; 

selectedLateralTargetError -= 360; 
selectedLateralTarget = 

preselected LateralTarget; 
lateralTarget = selectedLateralTarget; 
lateralTargetError = 

selectedLateralTargetError; 

> 

public void applyRule07() { 
outputState = 0; 

selectedLateralTargetError += 360; 
selectedLateralTarget = 

preselected LateralTarget; 
lateralTarget = selectedLateralTarget; 
lateralTargetError = 

selectedLateralTargetError; } 


isNominal[0] == false && outputState[2] != C0NST_1 && 
outputState[2] == C0NST_2 && 
selectedLateralTargetError [180] > C0NST_179 && 
userPressesLateralTargetButton_sl_[l] == true && 
userPressesLateralHoldButton_s2_[0] == false && 
userPressesLNAVbutton_s3_[0] == false 


r 


A 


outputState = 0; 

selectedLateralTargetError += 360; 
selectedLateralTarget = preSelectedLateralTarget; 
lateralTarget = selectedLateralTarget; 
lateralTargetError = selectedLateralTargetError; 






isl\lominal[0] == false && outputState[2] != C0NST_1 && 
outputState[2] == C0NST_2 && 
selectedLateralTargetError [180] > C0NST_179 && 
userPressesLateralTargetButton_sl_[l] == true && 
userPressesLateralHoldButton_s2_[0] == false && 
userPressesLNAVbutton_s3_[0] == false 
outputState [2] != CONSTJL && 
outputState [2] == C0NST_2 && 
selectedLateralTargetError [180] > C0NST_179 && 
userPressesLateralTargetButton_s4[l] == C0NST_1 
userPressesLateralHoldButton_5[0] == CONST_0 && 
userPressesLNAVbutton_6[0] == CONST_0 && 




r 


outputState = 0; 

selectedLateralTargetError += 360; 
selectedLateralTarget = preSelectedLateralTarget; 
lateralTarget = selectedLateralTarget; 
lateralTargetError selectedLateralTargetError; 








results and challenges 


■ automatically generated 1 6 test cases for n= I 

■ discovered through unsatisfiable path constraints that some 
rules disable each other 

■ (HAI challenge) provide support for modeling semantics of user 
interface components such momentary vs. toggle switch 

■ (HAI challenge) define coverage criteria - for example related 
to covering modes; also what values should we pick for n (what 
length of user inputs)? 

■ (generic challenge) scalability of symbolic execution 
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Generic 




thank you! 

dimitra.giannakopoulou@nasa.gov 

neha.s.rungta@nasa.gov 

michael.s.feary@nasa.gov 


symbolic execution for ADEPT HAI models 
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